Vector representation based on a supervised codebook for Nepali documents classification

Document representation with outlier tokens exacerbates the classification performance due to uncertain orientation of such tokens. Most existing document methods in different languages including Nepali mostly ignore strategies filter them out from documents before learning their representations. In this article, we propose a novel method based on supervised codebook represent documents, where our contains only semantic without outliers. Our is domain-specific as it given corpus that have higher similarities class labels corpus. adopts simple yet prominent for each word, called probability-based word embedding. To show efficacy method, evaluate its task using Support Vector Machine and validate against widely used Bag Words, Latent Dirichlet allocation, Long Short-Term Memory, Word2Vec, Bidirectional Encoder Representations Transformers so on, four text datasets (we denote shortly A1, A2, A3 A4). The experimental results produces state-of-the-art (77.46% accuracy 67.53% 80.54% 89.58% A4) compared methods. It yields best three (A1, A2 A3) comparable fourth dataset (A4). Furthermore, introduce largest (A4), NepaliLinguistic dataset, linguistic community.